

©008 



02/13/03 11:25 FAX 503274462 



MARGER JOHNSON McCOLLOM 



REMARKS 



Applicants and their representative would like to thank the Examiner for taking the 
time to speak with them on Wednesday, February 12, 2003, in an Examiner interview. 

Claims 1-12 and 14 are pending in the above-referenced application, claim 13 having 
been canceled by a previous amendment- 
Claim 8 was rejected under 35 USC §1 12 as containing subject matter which was not 
described in the specification in such a way as to reasonably convey to one skilled in the 
relevant art that the inventors, at the time the application was filed, had possession of the 
claimed invention. Specifically, the Examiner states that ". . .the specification explicitly 
discloses that a 'user may access the short-term memory via the audiovisual user navigation 
interface'..." 

However, the specification discloses that "A search engine 50 can be used via the 
audiovisual user navigation module interface 48 for information retrieval from long-term 
storage [emphasis added]," at page 15, lines 8-10. The user navigation module interface can 
retrieve information from both short-term storage and long-term storage. Having shown that 
the subject matter of claim 8 is described in the specification, Applicants respectfully request 
withdrawal of this objection. 

Claim 1 was rejected under 35 USC § 102(e) as being anticipated by Ramaswamy (US 
Patent No. 6,295,647). Applicants respectfully disagree. 

The Examiner states that Ramaswamy references "teaches that the context editor is 
utilized to 'identify key clips of audiovisual information' wherein the instant application 
broadly defines key clips as including other relevant information. . ." However, as amended, 
claim 1 specifically defines key clips as containing one of either key events or key objects as 
defined by a viewer. This is supported in Applicant's specification at page 5, line 20 through 
page 6, line 19, among other places. The reference to authoring key clips is supported at page 
10, lines 4 through 17, among other places. 
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As the executable code segments inserted in the video broadcast in Ramaswamy are 
not the same as the key clips upon which the key-clip data is based, Applicants submit that 
claim 1 is patentably distinguishable over the prior art and request allowance of this claim. 

Claims 1 and 7 are rejected under 35 USC § 102(e) as being anticipated by Srinivasan, 
et al. (US Patent no. 6,357,042). Applicants respectfully disagree. 

Srinivasan is directed to an authoring system for inserting annotations into video 
streams, such as text, thumbnails, etc. The Examiner states that Srinisvasan includes 'key 
clip data* of a defined metadata format. However, the metadata that is added to the video is 
not 'key clip* data, with regard to how key clips are now defined in Applicants' claim 1. 
Further, the system of Applicants' amended claim 1 requires that the data service authoring 
subsystem author the key clips, which is not shown, taught nor suggested by Srinisvasan. 
Srinivasan merely addresses adding annotations and identifying events in a video broadcast 
by adding information. In addition, Srinisvasan does not transmit the data broadcast service 
in such a manner that allows the user to customize it at the client receiver. 

Further Srinivasan does not use any descriptors in MPEG PSD? to correspond to the 
key clips of the instant application. Srinivasan makes a general reference to MPEG, but does 
not teach, show nor suggest the use of the PSIP descriptor constructs in the metadata, nor 
does Srinivasan teach that there is a direct correlation between the descriptors used and the 
PSIP descriptors. Therefore, Applicants submit that claim 1 is patentably distinguishable 
over the prior art and request allowance of this claim. 

With regard to claim 7, Applicants submit that claim 7 should be ruled allowable for 
the reasons as applied to claim 1. Additionally, Srinivasan does not show that the key clips, 
as that term is now defined in Applicant's claim 1, should be identified including their 
content. Srinivasan does include events, such as touchdowns (see column 28, lines 65-67), it 
does not include a defined set of key clips of either key events or key objects. The events 
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documented and annotated in Srinivasan are any event that occurs, not a defined key event or 
key object, nor does Srinivasan coordinate descriptors from MPEG PSIP data to descriptors 
in the key clip data. Therefore, Applicants submit that claim 7 is patentably distinguishable 
over the prior art and request allowance of this claim. 

Claims 2-6 are rejected under 35 USC §103(a) as being unpatentable over Srinivasan, 
et al. (US Patent No. 6,357,042). Applicant respectfully disagrees. 

With regards to claims 2-3 and 5-6, Srinivasan does not disclose the use of MPEG-2 
presentation time stamps for referencing key-clips as the term key-clips is defined in claim 1, 
from which these claims depend. The advantage of using MPEG-2 reference methods is in 
the standardization, where any receiver/client that is KdPEG-2 compliant can utilize the time 
references as discussed on page 10, line 16 through page 11, line 16; page 12, line 10 through 
line 22; and page 13, line 13, through line 23, depending upon the situation in which the 
video content was provided. Srinivasan mentions using a presentation time stamp that could 
be any type of presentation time stamp, not the MPEG-specific time stamps and constructs 
claimed by Applicant. Therefore, Applicant submits that claims 2-3 and 5-6 are patentably 
distinguishable over the prior art and request allowance of this claim. 

With regard to claim 4, the Examiner states that it would be obvious to incorporate a 
video reference generator for the purposes of referencing video frames to real time PCR 
information. However, as described in Applicants* specification at page 12, lines 4 through 
22, the video reference generator generates references to key clips, not just 'video frames.* 
As Srinivasan does not use key-clips as defined in Applicants* claim 1, nor does Srinivasan 
use the MPEG-2 PCR construct, Applicants submit that claim 4 is patentably distinguishable 
over the prior art and request allowance of this claim. 
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Claims 8-11 are rejected under 35 USC § 103(a) as being unpatentable over Sezan, et 
al. (US Patent No. 6,236,395) in view of Barton, et al. (US Patent No. 6,233389). Applicant 
respectfully disagrees. 

Claim 8 requires that the receiver have both a short-term memory for storage of the 
summaries and long-term storage of both programs and summaries. Storage of the 
summaries in short-term storage versus long-term storage has an advantage in that short-term 
storage is generally faster than long-term storage, as there is less content to search through in 
summaries than if there were also archived programs and their related information. Sezan 
does not disclose the flexibility of having both short-term storage and long-term storage. 
Given the advantage of having both, including the flexibility of the dual nature of the 
memory, it would not be obvious to include both. 

Further, the Sezan reference is directed to a descriptor scheme that does not have a 
summarizer with an inference engine that also extracts key clips using references in the 
broadcast. The Examiner directs the Applicants to the analysis module 42 and the description 
scheme generation module 44 and the text at column 8, lines 21-67. The referenced text 
depicts the operation of the analysis module to generate summaries using several sources of 
information. However, there is no reference to extracting key clips using the references 
supplied in the broadcast, as the summarizer is operable to do in Applicants' claim 8. As the 
extraction using references provided has several advantages over having to generate these 
summaries using detailed video analysis techniques, Applicants submit that claim 8 is 
patentably distinguishable over the prior art and request allowance of this claim. 

During the interview, the Examiner referred again to the analysis module of the Sezan 
reference as being equivalent to the inference engine. Applicants believe the Examiner may 
be confused as to the inputs to the analysis module of the Sezan reference. In the Sezan 
reference, the audiovisual broadcast information coming into the receiver 38 is relatively 
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untouched. The receiver also receives all of the auxiliary information such as web site 
information, EPG, etc., and passes these data streams to the analysis module, which in turn 
produces a data stream of the audiovisual program melded with all of the supplementary 
information. Applicants* invention, as claimed in claim 8, operates on a data stream that is 
equivalent to what comes out of the analysis module 42, not what comes out of the receiver 
38. There is no structure in Sezan that operates further on the output of analysis module 42 
that performs any functions or has the capabilities of the inference engine of Applicants* 
invention. 

Further, Applicants* invention as claimed in claim 8 require that inference engine to 
extract the key-clip data and the PSIP information in which there is a direct correspondence 
between the two. For these reasons, Applicants submit that claim 8 is patentably 
distinguishable over the prior art and request allowance of this claim. 

Claims 9-11 depend from claim 8 and should be ruled allowable for this reason and 
for their own merit. With regard to claim 9, Sezan and Barton in combination do not show a 
receiver with a decoding and demultiplexing module to produce program-related information, 
as program-related information is defined in claim 8, for within program filtering of 
audiovisual programs, where the receiver also includes both short-term and long-term 
storage. While the claims are construed in light of the specification, if the Applicant provides 
a specific definition in the specification, the words of the claim must be given that definition 
In re Tletz, 893 F.2d 319, 321, 13 USPQ2d 1320, 1322 (Fed Cir. 1989), In re Vogel 422 R2d 
438, 441, 164 USPQ 619, 622 (CCPA 1970). Applicants have defined within-program 
filtering to be based upon key events and key objects (Specification, page 5, lines 11-14), and 
Sezan and Barton do not show such a definition for within-program filter. Therefore, 
Applicants submit that claim 9 is patentably distinguishable over the prior art and request 
allowance of this claim. 
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With regard to claim 10, Sezan and Barton do not show that program-related 
information further comprises description information used as indices for archival of 
audiovisual programs in conjunction with long-term storage and short-term storage. More 
than likely, the storage of program-related information for database indexing would occur in 
the long-term storage. However, there is no analogy to such long-term storage in Sezan, as 
discussed above. The combination, in Sezan, merely mentions a storage unit and a filtering 
and browsing module that may be used to store/archive programs. There are several manners 
in which the Sezan description scheme could implement storing and archiving. There is no 
mention of a database, or using a description scheme as the indices for a database as is 
claimed in claim 10. Therefore, Applicants submit that claim 10 is patentably distinguishable 
over the prior art and request allowance of this claims. 

With regard to claim 11, Sezan and Barton do not show a receiver where a register of 
user preferences used to create summaries that can be stored in both long-term and short-term 
storage. Further, the combination does not show a receiver where user preferences are used 
not only by a summarizer module but also by the decoding and demultiplexing module to 
account for user preferences at the intake of an MPEG-2 encoded audiovisual program with 
its associated PSBP data and references to key-clips. The combination does not show 
extraction of the key-clips based upon the references in the broadcast, which has several 
advantages as point out above. Therefore, Applicants submit that claim 1 1 is patentably 
distinguishable over the prior art and request allowance of this claim. 

Claims 12 and 14 are rejected under 35 USC § 103(a) as being unpatentable over 
Sezan, et al. 

The Examiner states that analysis module of Sezan is equivalent to the program 
summarizer that is operable to receive a data broadcast filtering and generating summaries of 
audiovisual programs. As has been discussed at length above, the inference engine of the 
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instant application would operate on the output of the analysis module, as the existence of a 
data broadcast similar to the output of the analysis module is already assumed in the instant 
application. Further, the analysis module of Sezan does not extract key-clip data and PSIP 
information where those two have descriptors within each that directly correspond to each 
other. 

Applicants are confused as the Examiner's reference to Casey, as the structures set 
forth in the claim are not shown in Sezan, as discussed with regard to claim 8, and there is no 
mention in Sezan of adapting them to perform the tasks of the instant application and 
therefore must have resulted in a structural difference in the claimed invention. Further, there 
must be some suggestion in the cited reference to produce the claimed invention. In re Fine, 
837F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988). The Sezan reference is very specific in 
what the program summarizer is capable of, and there is no mention, reference or suggestion 
of that having a description extraction module or an inference engine. 

Additionally, Component 76 in Sezan is described as a key-frame summarizer. See 
col. 8, line 54. A component that produces key-frame summaries is not the same as a key 
clip extraction module that extracts key clips from an audiovisual program. Applicants had 
mis-named the key-clip extraction module a key-clip extraction table in a typographic error. 
However, in referring to Figure 5, it can be seen that the component is a key-clip extraction 
module. If the Examiner's position were that a key-frame summary is the same as a key-clip, 
then it would follow that the summaries of Sezan are vastly different than the summaries of 
the Applicants* invention as claimed. If the Examiner acknowledges that they are different, 
then the Sezan reference does not teach nor suggest a key clip extraction module. Applicants 
therefore submit that claim 12 is patentably distinguishable over the prior art. 

Claim 14 depends from claim 12. Applicants submit that claim 14 is patentably 
distinguishable over the prior art for the reasons as applied to claim 12. Further, the Sezan 
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reference does not teach, show nor suggest an inference engine, much less one that uses 
information downloaded from a web site in its operation. 

The Examiner concludes that the prior art made of record and not relied upon is 
considered pertinent to applicant's disclosure. DeVres discloses an automated system for 
performing annotations based upon probabilities and associated values within a media stream. 
Applicants' invention as claimed is directed to identifying key clips and providing references 
to those key clips to the receiver. While possibly in the same field of endeavor, this reference 
is not related to Applicants' invention as claimed. 

The Altunbasak reference describes a system for object-based video retrieval. This 
reference does not disclose a data service provider or receiver as claimed by Applicant. It is 
possible that such a system could be used at the provider end in conjunction with Applicants' 
invention as claimed, but is not relevant to Applicants' invention. 

The Orr reference deals strictly with text extraction and does not contemplate nor 
discuss extraction of key clips, providing references to key clips or many of the other 
components that contribute to Applicants* invention as claimed. 

The Carrer reference describes a system for annotating video data, not one for 
extracting key clips and providing program-related information to clients for use in extracting 
and storing meaningful summaries of the content, as does Applicants' claimed invention. 
Therefore, Applicant submits that the prior art cited but not relied upon is not relevant to 
Applicant's disclosure- 
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No new matter has been added by this amendment. Allowance of all claims is 
requested. 

Respectfully submitted, 

MARGER JOHNSON & McCOLLOM, P.C. 

>lk I ~BJ 

Julie Reed 

Reg. No. 35, 349 
MARGER JOHNSON & McCOLLOM 
1030 SW Morrison Street 
Portland, OR 97205 
(503) 222-3613 
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VERSION WITH MARKINGS TO SHOW CHANGES MADE 
1. (Thrice amended) A system for providing a digital television data broadcast service, 
comprising: 

a data service authoring subsystem operable to receive an audiovisual program and to 
author key clips, wherein key clips are comprised of one of either at least one key event or at 
least one key object, and provide key clip data in a defined format identifying the key clips of 
the audiovisual program 

a data service encoder operable to receive said key clip data and Program and System 
Information Protocol dat a, wherein key clip data includes descriptors that directly correspond 
to descriptors in the Pr ogram and System Information Protocol data, and finalize contents of 
said data broadcast service; and 

an MPEG-2 system multiplexer operable to multiplex said contents of said data 
broadcast service with encoded audiovisual programs and produce a MPEG-2 transport 
stream to be broadcast to at least one client receiver in a format that allows customization at 
the receiver . 

8. (Thrice amended) A receiver operable to receive and operate upon a digital television 
data broadcast service, comprising: 

a demultiplexing and decoding module to extract program-related information, 
wherein the program-related information further comprises an MPEG-2 encoded audiovisual 
program, Program and System Information Protocol data, and references to key-clips from 
the digital television data broadcast service; 

a summarizer operable to receive the audiovisual program PSIP data and references to 
key-clips and to create summaries of the audiovisual progra m, using the references to kev> 
clips to extract the key-clips from the program, wherein the summarizer includes an inference 
engine operable to combine said audiovisual program description with said PSIP information. 
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wherein descriptors in the audiovisual program description directly correspond to descriptors 
in the PSIP information, user preferences, and any other available program information to 
produce program-related information and kev-clip information : 

a navigation module operable to allow a user to browse said program-related 
information; 

a short-term memory to allow short-term storage of [at least one of the programs and] 
the summaries; and 

a long-term memory to allow long-term storage of the programs and the summaries, 
wherein the long-term storage is accessible from the navigation module. 
12. (Thrice amended) A program summarizer operable to receive a data broadcast service 
for filtering and generating summaries of audiovisual programs, comprising: 

a description extraction module operable to parse and extract an audiovisual program 
description provided by said data broadcast service; 

a program and system information extraction module operable to extract the program 
and system information protocol (PSIP) information and MPEG-2 System Information from 
said data broadcast service; 

an inference engine operable to combine said audiovisual program description with 
said PSIP information, where descriptors in the audiovisual program information directly 
correspond to descriptors in the PSIP information, u ser preferences, and any other available 
program information to produce program-related information and key-clip information; 

a key-clip map table operable to take said key-clip information and produce a map of 
video references and times; 

a key clip extraction module operable to extract key clips from said audiovisual 
progra m using references to the key clips in the data broadcast service : and 
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a summary composition module operable to produce summaries of said audiovisual 
program and provide it to a viewer. 
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